Two-Step Shortlisting by Imperfect Experts

We explore various models of decision making using an intermediate shortlisting step. The shortlisting and the final decision are done by two imprecise experts who may or may not be identical and/or independent. MSC 2000: Primary 91B06, Secondary 90B50 Introduction Decision-making processes often consist of more than one step: I Junior management or consultants analyze a large number of possible approaches to a problem and present a short list of alternatives. Senior management then reviews this list and makes a final choice or decision. I Computerized Decision Support Systems such as Recommender Systems (Resnick & Varian, 1997) make a small set of recommendations from which the end user selects one course of action. I Chess professionals use the k-best mode of computer chess programs and make the final choice themselves when analyzing games or researching new variants (see Bareev, 2001). These examples are instances of “division of labor”-approaches for making decisions based on two-step algorithms. One implicit assumption of such a procedure is that the sequential application of more than one expert improves the final decision. This assumption is rarely explicitly acknowledged. In fact, real world experts are not perfect. They cannot measure hidden costs and benefits of a course of action and need to reduce complexity to be able to handle reality. We analyze some two-step shortlisting models with imperfect experts using Monte Carlo simulations and theoretical approaches.